Image processing apparatus, image processing method, and storage medium

ABSTRACT

An image processing apparatus includes: a designating unit configured to designate a color; a reading unit configured to read a document to generate a document image; a conversion unit configured to convert a color of a pixel in the document image to a specific color based on a designated color designated by the designating unit; and a specification unit configured to specify a graphic region in the document image. The conversion unit is configured not to convert a color of a pixel in the graphic region specified by the specification unit to the specific color.

BACKGROUND OF THE INVENTION Field of the Invention

The present disclosure relates to an image processing apparatus, animage processing method, and a storage medium.

Description of the Related Art

Although there has been progress in digitization, there are still manyoccasions today where characters and symbols are written on printedmaterials (e.g., conference handouts) with writing instruments. However,when later duplicating and sharing the materials with others, it may bedesirable to erase handwritten content. In order to satisfy such adesire, it has been proposed to incorporate, in apparatuses such asscanners, copiers, and multi-function peripherals, a so-called colordropout function for removing, from the image data of an optically-readdocument, a color designated by a user (refer to Japanese PatentLaid-Open No. 2011-188484 and Japanese Patent Laid-Open No.2016-139868).

The color dropout function proposed in Japanese Patent Laid-Open No.2011-188484 allows the user to designate, on a setting screen fordisplaying a color plane, a range of colors that should be removed andconverts colors in image data that fall within that range to white usinga look-up table method. However, a technique that uniformly removesdesignated colors also removes the same colors in the non-handwrittenportion, thereby reducing the image quality of a read image. Forexample, when an attempt is made to erase characters written with a redpen on a material, a red subject in a photograph inserted in thematerial can also end up being erased.

Japanese Patent Laid-Open No. 2016-139868 proposes, in order to avoidunintentional deterioration in image quality of photographic regions,determining halftone dot regions, which have a gradation representation,in image data as photographic regions, and not performing color removalin the photographic regions.

SUMMARY OF THE INVENTION

However, the photographic regions are not the only regions in whichdeterioration in image quality caused by color removal becomes aproblem. For example, multiple colors may be used for graphic elementssuch as graphs and figures, which are often inserted into conferencematerials. However, since sub-regions of the graphic elements havealmost uniform densities when viewed locally, they are not determined asphotographic regions with the determination method based on thegradation representation proposed in Japanese Patent Laid-Open No.2016-139868 and will not be excluded from being a color removal target.The user may also not wish to remove color in sub-regions of those otherthan the regions of graphic elements or may wish to remove color only ina specific sub-region.

Thus, there is still a need for a mechanism that can remove a designatedcolor from image data of a document while preventing an undesirabledeterioration in image quality.

According to an aspect, there is provided an image processing apparatusincluding: a designating unit configured to designate a color; a readingunit configured to read a document to generate a document image; aconversion unit configured to convert a color of a pixel in the documentimage to a specific color based on a designated color designated by thedesignating unit; and a specification unit configured to specify agraphic region in the document image; wherein the conversion unit isconfigured not to convert a color of a pixel in the graphic regionspecified by the specification unit to the specific color.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a basicconfiguration of a multi-function peripheral.

FIG. 2 is an explanatory diagram for explaining the disadvantages of aconventional color dropout function.

FIG. 3 is a block diagram illustrating an example of a logicalconfiguration of a multi-function peripheral according to a firstembodiment.

FIG. 4 is an explanatory diagram for explaining an example of a settingGUI that can be displayed on a screen in the first embodiment.

FIG. 5 is an explanatory diagram for explaining a technique fordetermining a graphic region according to the first embodiment.

FIG. 6 is a sequence diagram illustrating a flow of basic processingperformed by the multi-function peripheral together with interactionsbetween the user and the multi-function peripheral.

FIG. 7A is a flowchart illustrating an example of a flow of imageprocessing according to the first embodiment.

FIG. 7B is a flowchart illustrating an example of a detailed flow of aregion determination process according to the first embodiment.

FIG. 8 is a block diagram illustrating an example of a logicalconfiguration of a multi-function peripheral according to a secondembodiment.

FIG. 9 is an explanatory diagram for explaining an example of a settingGUI that can be displayed on a screen in the second embodiment.

FIG. 10 is a flowchart illustrating an example of a detailed flow of aregion setting process according to the second embodiment.

FIG. 11 is a block diagram illustrating an example of a logicalconfiguration of a multi-function peripheral according to a thirdembodiment.

FIG. 12 is an explanatory diagram for explaining an example of a settingGUI that can be displayed on a screen in the third embodiment.

FIG. 13 is a flowchart illustrating an example of a detailed flow of aregion setting process according to the third embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention. Multiple features aredescribed in the embodiments, but limitation is not made to an inventionthat requires all such features, and multiple such features may becombined as appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

1. Introduction

In this specification, examples in which a technique according to thepresent disclosure is applied to a multi-function peripheral (MFP) willbe mainly described. However, the technique according to the presentdisclosure is not limited to such examples and is widely applicable toimage processing apparatuses in general (including, for example,copiers, printers, scanners and facsimile machines). Also, unlessotherwise specified, each of the components to be described below, suchas apparatuses, devices, modules and chips, may consist of a singleentity or a plurality of physically different entities.

<1-1. Basic Configuration of Apparatus>

FIG. 1 is a block diagram illustrating an example of a basicconfiguration of a multi-function peripheral. A multi-functionperipheral 100 illustrated in FIG. 1 includes a CPU 101, a ROM 102, aRAM 103, a storage unit 104, an operation unit 105, a scanner 111, aprinter 112, a communication I/F 113, an image processing unit 115, anda bus 120. The CPU (Central Processing Unit) 101 is a processor forcontrolling the general functions of the multi-function peripheral 100.The ROM (Read Only Memory) 102 is a non-volatile memory for storing, inadvance, a computer program to be executed by the CPU 101. The ROM 102may be, for example, a flash memory. The RAM (Random Access Memory) 103is a volatile memory providing the CPU 101 with a work storage region.When the computer program is executed by the CPU 101, for example, theRAM 103 temporarily stores program data, control data, and image data tobe processed. The storage unit 104 is utilized as a so-called auxiliarystorage device and can store image data of many images and other data.The storage unit 104 may be, for example, an HDD (Hard Disk Drive). Theoperation unit 105 provides a user interface to a user of themulti-function peripheral 100. The operation unit 105 includes, forexample, a display device for displaying information and images and aninput device for receiving user inputs. The scanner 111 is a readingunit that generates image data by reading a document. Depending on thesetting of the reading, the image data generated by the scanner 111 canbe color image data or monochrome image data. The printer 112 is animage forming unit that forms an image on a printing medium based onimage data designated in a job. In copy mode, after the image datagenerated by the scanner 111 has been processed by the image processingunit 115 to be described later, an image based on the processed imagedata is formed by the printer 112 (that is, a document is copied). Inprint mode, an image based on the image data designated by a user isformed on a printing medium by the printer 112 (i.e., an image isprinted). The communication interface (I/F) 113 is an interface for themulti-function peripheral 100 to communicate with other apparatuses viaa network. The image processing unit 115 is a processing circuit forperforming a variety of image processing, such as color spaceconversion, size change, and noise reduction, for example, on image dataoutputted from the scanner 111 and image data inputted to the printer112. The functions such as region determination and color removal to bedescribed in connection with the embodiments to be described below maybe implemented by one of the CPU 101 and the image processing unit 115or by both cooperating with each other. The bus 120 is a signal linethat connects the CPU 101, the ROM 102, the RAM 103, the storage unit104, the operation unit 105, the scanner 111, the printer 112, thecommunication I/F 113, and the image processing unit 115 to each other.

<1-2. Explanation of Problems>

On the left in FIG. 2, an appearance of a document 210 is illustrated asan example of something to be read by the scanner 111. The document 210is, for example, a conference handout and includes, in addition to text,graphic elements 211 and 212. The graphic element 211 is a pie chart,and graphic element 212 is a bar graph. In addition, characters 213 andsymbols 214 and 215 have been handwritten on the document 210. Assumethat the handwritten portions are red and some of the graphic elements211 and 212 have also been depicted in red. On the right in FIG. 2, anoutput image 220 for when, in a conventional color dropout function, redhas been designated as the color that should be removed is illustrated.In the output image 220, the characters 213 and symbols 214 and 215 havebeen successfully removed, but the red portions of the graphic elements211 and 212 are missing against the user's intention.

As will be described in more detail in the next section, the techniqueaccording to the present disclosure provides a mechanism capable ofappropriately removing designated colors from the image data of adocument while preventing such deterioration in image qualityundesirable to users.

2. First Embodiment

<2-1. Functional Configuration>

FIG. 3 is a block diagram illustrating an example of a logicalconfiguration of the multi-function peripheral 100 according to a firstembodiment. Referring to FIG. 3, the multi-function peripheral 100includes a data acquisition unit 301, a setting unit 302, a regiondetermination unit 303, and a removal unit 304.

(1) Data Acquisition Unit

The data acquisition unit 301 acquires color image data to which a colordropout function of the present embodiment is to be applied. Forexample, the data acquisition unit 301 may cause the scanner 111 to reada document and then acquire color image data generated by the scanner111. The data acquisition unit 301 may acquire color image data storedin the storage unit 104 or color image data received from anotherapparatus via the communication I/F 113. The data acquisition unit 301outputs the acquired color image data to the region determination unit303 and the removal unit 304.

(2) Setting Unit

The setting unit 302 acquires, via the operation unit 105, settinginformation related to removal of color from color image data and setsthe acquired setting information in the removal unit 304. The settinginformation related to color removal includes at least a designatedcolor to be removed from the color image data. For example, the settingunit 302 displays, on the screen of the operation unit 105, a settingGUI (Graphical User Interface) for accepting settings related to colorremoval and acquires the setting information based on user inputs to thesetting GUI. FIG. 4 illustrates an example of a setting GUI that can bedisplayed on the screen of the operation unit 105 in the presentembodiment.

A main menu 410 on the upper left in FIG. 4 is a menu for displaying aselection of the main functions of the multi-function peripheral 100.The main menu 410 includes buttons 411, 412, 413, 414, and 415. Thebutton 411 is a button for invoking a “scan and transmit” function. Thebutton 412 is a button for invoking a “scan and save” function. Thebutton 413 is a button for invoking a “box” function (for using datastored in a storage device). The button 414 is a button for invoking a“copy” function. The button 415 is a button for invoking a “print”function. The mechanism of color removal to be described in thisspecification is applicable to any of these functions, but here, anexample in which it is applied to the “copy” function will be described.When the user operates (e.g., touches) the button 414, the GUItransitions from the main menu 410 to a copy setting menu 420. The copysetting menu 420 includes, for example, color mode (fullcolor/monochrome), magnification, paper size, and number of copies, asbasic setting items. The copy setting menu 420 also includes, forexample, density, single-sided/double-sided, document type, and othersettings, as advanced setting items. When the user selects “full color”as the color mode and operates a button 421 (“other settings”), the GUItransitions from the copy setting menu 420 to another settings menu 430.The other settings menu 430 includes buttons for invoking variousadvanced settings. For example, when the user operates a button 431(“color removal”), the GUI transitions from the other settings menu 430to a color removal setting menu 440.

The color removal setting menu 440 includes buttons 441, 442, 443, and444 respectively corresponding to a plurality of colors that can beselected as colors to be removed. The button 441 is a button forselecting red. The button 442 is a button for selecting green. Thebutton 443 is a button for selecting blue. The button 444 is a buttonfor selecting black. By operating a plurality of buttons, two or morecolors may be selected as colors to be removed at the same time. Notethat the candidates for the colors indicated here are only examples.More, fewer, or different colors can be selected as candidates forcolors to be removed. Note that the settings related to color removalaccepted via the setting GUI is not limited to the example of FIG. 4 andmay include further setting items. Further setting items may include,for example, one or more of a range of brightness that should beremoved, a range of density of color that should be removed, anddesignating a background color. The setting unit 302 sets, in theremoval unit 304, the setting information related to color removalacquired via the color removal setting menu 440.

(3) Region Determination Unit

The region determination unit 303 determines or specifies, in the colorimage data inputted from the data acquisition unit 301, a removal regionin which color removal should be executed and a non-removal region whichshould be excluded from being a color removal target. In thisembodiment, the non-removal region includes at least graphic regions.The graphic regions correspond to regions in which a graphic element asdescribed with reference to FIG. 2 is depicted. The region determinationunit 303 may determine photographic regions in color image data inaccordance with a known photographic region determination technique andtreat both the graphic regions and the photographic regions as thenon-removal region. In the present embodiment, regions that have notbeen determined as the non-removal region will be the removal region.

While scanning pixels in color image data, which is a two-dimensionalpixel array, in raster scan order, for example, the region determinationunit 303 sets, as a determination window, an M-row-by-N-column pixelgroup including a pixel of interest, with each pixel as the pixel ofinterest. Here, M and N are integers of 2 or more. Then, the regiondetermination unit 303 determines whether a determination condition thatvariance in color in the determination window is lower than apredetermined criterion is satisfied. When the variance in color in thedetermination window is lower than the predetermined criterion, theregion determination unit 303 determines that the pixel of interestbelongs to a graphic region. Generally, a local region of a graphicelement is usually rendered on a printing medium at a uniformconcentration (i.e., as a solid region). However, when read by thescanner 111, the density may vary between pixels in the read image datadepending on the read accuracy and noise. Therefore, determining, asgraphic regions, regions in which a color is not completely uniform butthe variance in color is lower than the predetermined criterion makes itpossible to appropriately determine the graphic regions.

As an example, the variance in color in the determination window beinglower than the predetermined criterion includes, for any color componentvalue of any pixel of the determination window, a difference betweenthat color component value and a corresponding average color componentvalue calculated over the entire determination window being lower than adetermination threshold. For example, portions handwritten influorescent ink or pencil have a tendency for density unevenness foreach pixel to be relatively large. Therefore, determining whether acolor component exhibiting a large deviation from an average colorcomponent value is present within the determination window in accordancewith the above determination method makes it possible to accuratelydistinguish between such handwritten portions and graphic regions. Also,for example, although the density unevenness in portions handwrittenwith a ballpoint pen is not very large, characters written with penusually have narrow line widths. Therefore, using a determination windowof a size that is larger than such narrow line widths (a determinationwindow of a relatively large area) makes it possible to accuratelydistinguish characters written with pen from graphic regions. Thus, theaccuracy in determining the graphic region is increased. Note that colorcomponents in the above determination method may be any color componentsconstituting any color space, such as RGB, CMY, or YUV, for example.

When the variance in color for a certain determination window is lowerthan the above-described criterion, the region determination unit 303may determine that not only the pixel of interest but also theneighboring pixels included in the determination window belong to agraphic region. Then, for pixels once determined to belong to a graphicregion, the region determination unit 303 may maintain the determinationthat the pixels belong to a graphic region regardless of the variance incolor in other determination windows. That is, when two pixel groups(determination windows) whose pixels of interest are different includesome pixels in common, if it is determined that the common pixels belongto a graphic region at least once during scanning, it is determined, asthe final determination result, that those pixels belong to a graphicregion. This avoids conflicting determination results between two ormore determination windows that overlap each other at the boundaryportions of the graphic elements and allows consistent graphicdetermination.

FIG. 5 is an explanatory diagram for explaining a graphic regiondetermination method in the present embodiment. The document 210illustrated in FIG. 2 is illustrated in a magnified manner in the upperportion in FIG. 5 (however, the upper half of the document 210 isomitted). On the lower left and lower right in FIG. 5, pixel groups,each included in a determination window when the scanning by the regiondetermination unit 303 reaches a position 501 of the read image data(color image data) of the document 210, are illustrated with theirrespective pixel values. However, the position of a lower rightdetermination window 520 is advanced by one pixel with respect to alower left determination window 510. Here, N=M=3 as an example, andthus, each determination window consists of nine pixels. The threevalues separated by commas illustrated inside a square of each pixel inthe figure are color component values (R, G, B) of that pixel in an RGBcolor space.

The pixel group of the determination window 510 includes a pixel ofinterest 511 located at the center, and eight neighboring pixels to theupper left, above, upper right, left, right, lower left, below, andlower right of the pixel of interest 511. Average color component values(R_(AVE), G_(AVE), B_(AVE)) calculated over the entire determinationwindow 510=(251, 41, 10). In addition, a color component value of apixel within the determination window with the largest difference fromthe corresponding average color component value is a B component of thepixel to the lower left of the pixel of interest 511, and the difference(absolute value) is equal to 12 (=22−10). Here, when a determinationthreshold for graphic region determination is 30, the above differenceis lower than the determination threshold. Based on the result ofcomparison to the threshold, the region determination unit 303determines that the pixel of interest 511 and the eight neighboringpixels in the determination window 510 belong to a graphic region.

The pixel group of the determination window 520 includes a pixel ofinterest 521 located at the center, and eight neighboring pixels nearthe pixel of interest 521. The average color component values (R_(AVE),G_(AVE), B_(AVE)) calculated over the entire determination window520=(251, 111, 88). In addition, a color component value of a pixel inthe determination window with the largest difference from thecorresponding average color component value is a B component of thepixel to the right of the pixel of interest 511, and the difference(absolute value) is equal to 159 (=247−88) and is higher than thedetermination threshold (=30). Here, the neighboring pixels to the upperleft, above, left, lower left, and below the pixel of interest 521 andthe pixel of interest 521 have already been determined to belong to agraphic region based on the determination result for the determinationwindow 510. Therefore, the region determination unit 303 maintains thedetermination that these pixels belong to a graphic region. Meanwhile,the region determination unit 303 tentatively determines that theneighboring pixels to the upper right, right, and lower right of thepixel of interest 521 do not belong to a graphic region. The result ofthis tentative determination that a pixel does not belong to a graphicregion may be overwritten based on the result of a new determinationafter the scan has advanced.

The region determination unit 303 repeatedly executes theabove-described region determination while scanning the pixels in thecolor image data in order, for example. Then, the region determinationunit 303 registers the pixels determined to belong to a graphic regionin a memory as the pixels of the non-removal region (e.g., adding to acorresponding pixel list or marking pixels of the non-removal region).The pixels that have not been finally registered as pixels of thenon-removal region constitute the removal region and will be subject tocolor removal by the removal unit 304 to be described later.

Note that although FIG. 5 illustrates an example in which a pixel groupof 3×3 constitutes a determination window (i.e., M=N=3), the size andshape of the determination window are not limited to such an example. Insome variations, the parameters M and N, which affect the size of thedetermination window, may be set based on the reading resolution ofdocument reading in the scanner 111. For example, even if the widths ofthe handwritten lines are the same, if the reading resolution is higher,the number of pixels corresponding to a line width in the read imagedata will be larger. Therefore, increasing the size to which thedetermination window is set as the resolution at which the documentreading is performed is increased makes it possible to prevent apossibility that a handwritten portion may be erroneously determined tobe the non-removal region.

Further, although FIG. 5 illustrates an example in which the pixel ofinterest moves by one pixel in the scanning for region determination, inorder to speed up the scanning, the pixel of interest may be set every npixels in the row direction and every m rows in the column direction (mand n are integers; 1<m≤M, 1<n≤N).

(4) Removal Unit

The removal unit 304, by replacing, with a background color, a color ofa pixel indicating a color (hereinafter, referred to as designatedcolor) set by the setting unit 302 as something that should be removedfrom the color image data inputted from the data acquisition unit 301,removes the designated color or converts the designated color to aspecific color. However, in the present embodiment, the removal unit 304does not remove (does not convert to a background color) a color of apixel that has been determined to belong to the non-removal region bythe region determination unit 303. Here, the non-removal region includesat least the graphic regions. The background color may be predefined,for example, as white. Further, the removal unit 304 may treat a colordesignated by the user via the operation unit 105 as the backgroundcolor or treat a color that is automatically recognized in a backgroundregion of the color image data as the background color.

More specifically, the removal unit 304, while scanning the pixels inthe color image data, for example, in raster scan order, determineswhether each pixel belongs to the non-removal region determined by theregion determination unit 303. For pixels that do not belong to thenon-removal region, if the pixel values of the pixels indicate adesignated color, the removal unit 304 replaces the pixel values of thepixels with the pixel value of the background color. By repeating suchdetermination and replacement for all pixels in the removal region, itis possible to appropriately remove a color that should be removed fromthe image data of a document while preventing color removal that goesagainst the user's intention.

<2-2. Processing Flow>

(1) Overall Flow

FIG. 6 is a sequence diagram illustrating a flow of basic processingperformed by the multi-function peripheral 100 together withinteractions between the user and the multi-function peripheral 100.Note that, in the following description, processing steps will beabbreviated as S (step).

First, in S601, the operation unit 105 displays the main menu 410 on ascreen. In S602, the user selects a desired function by operating abutton of the main menu 410 corresponding to the desired function. Here,assume that the user selects the copy function. Next, in S603, theoperation unit 105 displays the copy setting menu 420, which is a typeof function menu, on the screen. In S604, the user inputs basic settings(e.g., color mode, magnification, paper size and number of copies) forthe copy function in the copy setting menu 420. The setting unit 302acquires the values of the basic settings accepted by the operation unit105 and then stores these in a memory. In addition, in S606, the useroperates the button 421 for “other settings” to perform advancedsettings (advanced settings). Next, in S607, the operation unit 105displays the other settings menu 430, which is a type of advancedsettings menu, on the screen. In S608, the user who desires to remove adesignated color when copying a document operates the button 431 toinvoke the color removal function. Next, in S609, the operation unit 105displays the color removal setting menu 440 on the screen. In S610, theuser selects settings related to color removal (e.g., one or more colorsthat should be removed) in the color removal setting menu 440. In S611,the setting unit 302 acquires the values of the settings related tocolor removal received by the operation unit 105 and sets the values tothe removal unit 304. When the required settings are completed, in S612,the user instructs the multi-function peripheral 100 to start executinga job by performing a predetermined operation (e.g., pressing a physicalstart key or touching a start button on the GUI). When an instructionfrom the user is detected by the operation unit 105, the CPU 101 causesthe scanner 111 and the printer 112 to start executing a copy job. InS613, the scanner 111 reads the document and generates color image data.Next, in S614, the CPU 101 and the image processing unit 115 executesimage processing including the above-described region determination andcolor removal for the generated color image data. In S615, theimage-processed image data is outputted to the printer 112, and an imagebased on the image data is formed on a printing medium by the printer112. Note that, when the user selects the scanning function instead ofthe copy function, the image-processed image data may be stored in adesignated location or transmitted to a designated destination insteadof an image being formed on a printing medium. In S616, the result ofexecuting the job is notified to the user on the screen.

(2) Image Processing Details

FIG. 7A is a flowchart illustrating an example of a flow of imageprocessing that may be performed by the multi-function peripheral 100 inS614 of FIG. 6. Here, the image processing can be realized, for example,by the CPU 101 executing a computer program loaded from the ROM 102 tothe RAM 103 in cooperation with the image processing unit 115.

First, in S701, the setting unit 302 acquires, via the operation unit105, setting information related to color removal including adesignation of a color that should be removed. The setting unit 302 setsthe acquired setting information in the removal unit 304. Next, in S703,the data acquisition unit 301 acquires the color image data of adocument. Then, the data acquisition unit 301 outputs the acquired colorimage data to the region determination unit 303 and the removal unit304. Next, in S705, the region determination unit 303 determines aremoval region and a non-removal region in the color image data byexecuting the region determination process for the color image data. Amore detailed flow of the region determination process performed herewill be described later. Next, in S731, the removal unit 304 selects onepixel in the color image data as a pixel of interest. Next, in S733, theremoval unit 304 determines whether or not the pixel of interest is apixel in the non-removal region based on the result of the regiondetermination process. Here, the non-removal region includes at leastthe graphic regions. When it is determined that the pixel of interest isnot a pixel in the non-removal region (is a pixel in the removalregion), in S735, the removal unit 304 determines whether the pixelvalue of the pixel of interest indicates the designated color. Forexample, when the pixel value (a three-dimensional vector comprisingthree color component values) of the pixel of interest is included in asub-space in a color space corresponding to the designated color, theremoval unit 304 may determine that the pixel value of the pixel ofinterest indicates the designated color. When it is determined that thepixel value of the pixel of interest indicates the designated color, instep S737, the removal unit 304 replaces the color of the pixel ofinterest with a background color. Meanwhile, when it is determined thatthe pixel value of the pixel of interest does not indicate thedesignated color, the removal unit 304 does not replace the color of thepixel of interest with the background color. If it is determined in S733that the pixel of interest is a pixel in the non-removal region, theprocessing steps, S735 and S737, will be skipped. Next, in S739, theremoval unit 304 determines whether unprocessed pixels remain in thecolor image data. If unprocessed pixels remain, the process returns toS731, the next pixel is selected as the pixel of interest, and S733 toS739 are repeated. When no unprocessed pixel remains, the imageprocessing in FIG. 7A ends.

(3) Region Determination Process Details

FIG. 7B is a flowchart illustrating an example of a detailed flow of aregion determination process that may be performed by the multi-functionperipheral 100 in S705 of FIG. 7A. Here, the region determinationprocess can be realized, for example, by the CPU 101 executing acomputer program loaded from the ROM 102 to the RAM 103 in cooperationwith the image processing unit 115.

First, in S711, the region determination unit 303 selects one pixel inthe color image data as a pixel of interest. Next, in S713, the regiondetermination unit 303 sets a determination window to a pixel groupincluding the pixel of interest. Next, in S715, the region determinationunit 303 calculates, for example, the average color component values ofrespective RGB color components over the entire determination window.Next, in S717, the region determination unit 303 determines whetherthere is a pixel with a color component value whose difference from thecorresponding average color component value is higher than thedetermination threshold in the determination window. If there is no sucha pixel in the determination window, in S719, the region determinationunit 303 determines that the pixels in the determination window belongto a graphic region. Meanwhile, if the difference between the colorcomponent value and the corresponding average color component value islower than the determination threshold for any color component value ofany pixel in the determination window, the processing step S719 will beskipped. Note that, although not illustrated in the drawing, the regiondetermination unit 303 may perform determination as to whether the pixelof interest belongs to another type of non-removal region (for example,a photographic region) in addition to the graphic region determinationin S719. Next, in S721, the region determination unit 303 determineswhether unprocessed pixels remain in the color image data. Ifunprocessed pixels remain, the process returns to S711, the next pixelis selected as the pixel of interest, and S713 to S721 are repeated.When no unprocessed pixel remains, in S723, the region determinationunit 303 determines that the pixels that have not been determined tobelong to a graphic region (and other types of non-removal region)belong to the removal region. Then, the region determination process ofFIG. 7B ends.

3. Second Embodiment

In the first embodiment, an example of treating graphic regions as thenon-removal region has been mainly explained. In this section, as asecond embodiment, an example in which a user is enabled to explicitlydesignate a non-removal region or a removal region on the screen will bedescribed.

<3-1. Functional Configuration>

FIG. 8 is a block diagram illustrating an example of a logicalconfiguration of the multi-function peripheral 100 according to thesecond embodiment. Referring to FIG. 8, the multi-function peripheral100 includes a data acquisition unit 801, a setting unit 802, and aremoval unit 804.

(1) Data Acquisition Unit

The data acquisition unit 801 acquires color image data of a document inthe same manner as the data acquisition unit 301 according to the firstembodiment and outputs the acquired color image data to the setting unit802 and the removal unit 804.

(2) Setting Unit

The setting unit 802 acquires setting information related to removal ofcolor from color image data via the operation unit 105 and sets theacquired setting information in the removal unit 804. Thecolor-removal-related setting information includes the designated colordesignated by the user as something that should be removed. In addition,in the present embodiment, the color-removal-related setting informationincludes the region information indicating the non-removal region or theremoval region designated by the user. For example, the setting unit 802displays a GUI for accepting the designation of a region on the screenof the operation unit 105 together with a preview image based on thecolor image data and then acquires the region information based on adetected user input.

FIG. 9 illustrates an example of a GUI that can be displayed on thescreen of the operation unit 105 in the present embodiment. In thesecond embodiment, when the user operates the button 431 (“colorremoval”) in the other settings menu 430 illustrated in FIG. 4, a colorremoval setting menu 940 illustrated on the upper left of FIG. 9 isdisplayed. The color removal setting menu 940 includes a check box 945in addition to the buttons 441, 442, 443, and 444 corresponding torespective color choices. The check box 945 is a UI for prompting theuser to select whether or not to display a preview at the start of a job(before printing is performed). When the execution of a copy job isstarted with “enable preview display” (“Yes”) selected, the operationunit 105 displays the preview image 951 based on the color image data ofthe document read by the scanner 111 on the screen 950. Then, theoperation unit 105 receives, on this screen, a designation of a regionby the user. For example, the region may be designated by a two-pointdesignation method in which two vertices of the upper left and lowerright of a rectangular region are designated by a drag operation or twotouch operations. In addition, the region may be designated by amulti-point designation method in which the vertices of a polygonalregion are sequentially designated by multiple touch operations. On thelower right of FIG. 9, a region 962 designated by the two-pointdesignation method and a region 963 designated by the multi-pointdesignation method are illustrated. The operation unit 105 displays anobject for masking each region over the preview image 951 so that thedesignated region is distinguishable by the user. Here, it is assumedthat the regions 962 and 963 are designated as non-removal regions,which will not be a target of execution of color removal. The settingunit 802 sets or specifies, as the color-removal-related settinginformation in the removal unit 804, the region information acquired viathe region designation on the preview image.

(3) Removal Unit

Based on the region information set or specified by the setting unit802, the removal unit 804 removes the designated color from the colorimage data inputted from the data acquisition unit 801 by replacing thecolor of the pixel indicating the designated color with the backgroundcolor. In a first example of the second embodiment, the regioninformation indicates the non-removal region(s) designated by the user.In this case, the removal unit 304 does not remove the designated colorin the non-removal region(s) and removes the designated color in theremaining region. In a second example of the second embodiment, theregion information indicates the removal region(s) designated by theuser. In this case, the removal unit 304 removes the designated coloronly in the removal region(s). As described above, the background colormay be predefined as white, for example; designated by a user; ordetermined automatically based on the inputted color image data.

More specifically, the removal unit 804, while scanning the pixels inthe color image data, determines whether each pixel belongs to a regionindicated by the region information. In the first example, for pixelsthat do not belong to any non-removal region indicated by the regioninformation, the removal unit 804 replaces the pixel values of thepixels with the pixel value of the background color when the pixelvalues of the pixels indicate the designated color. In the secondexample, for pixels that belongs to a removal region indicated by theregion information, the removal unit 804 replaces the pixel values ofthe pixels with the pixel value of the background color when the pixelvalues of the pixels indicate the designated color. By repeating suchdetermination and replacement, it is possible to appropriately removethe color that should be removed from the image data of the documentwhile reliably avoiding the execution of color removal that goes againstthe explicit intention of the user.

<3-2. Processing Flow>

The flow of the basic processing performed by the multi-functionperipheral 100 in the second embodiment may be roughly the same as theflow described with reference to FIG. 6. However, in the secondembodiment, after the document is read in step S613, at the beginning ofimage processing in S614, the display of the preview image and theacceptance of region designation are performed by the operation unit105.

The flow of the image processing that corresponds to S614 of FIG. 6 inthe second embodiment may be roughly the same as the flow described withreference to FIG. 7A. However, in the second embodiment, in S705, aregion setting process illustrated in FIG. 10 is performed by thesetting unit 802 instead of the region determination process illustratedin FIG. 7B. Further, in S733, it is determined whether or not the pixelof interest is a pixel in a non-removal region based on the regioninformation indicating the region designated by the user.

FIG. 10 is a flowchart illustrating an example of a detailed flow of aregion setting process that may be performed by the multi-functionperipheral 100 in the second embodiment. Here, the region settingprocess can be realized, for example, by the CPU 101 executing acomputer program loaded from the ROM 102 to the RAM 103 in cooperationwith the image processing unit 115.

First, in S1001, the setting unit 802 displays, on the screen of theoperation unit 105, a preview image (for example, a low-resolutionimage) based on the color image data of the document. Next, in S1003,the operation unit 105 receives designation of a region by the user inthe method described with reference to FIG. 9, for example. The regiondesignated here is either a non-removal region or a removal region.Then, in S1005, the setting unit 802 acquires the region informationindicating the designated region, and sets the acquired regioninformation in the removal unit 804. Then, the region setting process ofFIG. 10 ends.

4. Third Embodiment

In a third embodiment, it is determined whether or not to remove thecolor of each pixel based on both the result of determination of thegraphic region and explicit region designation by the user.

<4-1. Functional Configuration>

FIG. 11 is a block diagram illustrating an example of a logicalconfiguration of the multi-function peripheral 100 according to thethird embodiment. Referring to FIG. 11, the multi-function peripheral100 includes a data acquisition unit 1101, a setting unit 1102, a regiondetermination unit 1103, and a removal unit 1104.

(1) Data Acquisition Unit

The data acquisition unit 1101 acquires color image data of a documentin the same manner as the data acquisition unit 301 according to thefirst embodiment, and outputs the acquired color image data to thesetting unit 1102, the region determination unit 1103, and the removalunit 1104.

(2) Region Determination Unit

The region determination unit 1103 determines or specifies a non-removalregion which should be excluded from being a color removal target in thecolor image data inputted from the data acquisition unit 1101. In thisembodiment, the non-removal region includes at least a graphic region.The region determination unit 1103 may determine the graphic region inthe color image data in the same method as the region determination unit303 according to the first embodiment. The non-removal region mayfurther include a photographic region. The region determination unit1103 outputs the result of the region determination to the setting unit1102 and the removal unit 1104.

(3) Setting Unit

The setting unit 1102 acquires setting information related to removal ofcolor from color image data via the operation unit 105 and sets theacquired setting information in the removal unit 1104. Thecolor-removal-related setting information includes the designated colordesignated by the user as something that should be removed. In addition,in the present embodiment, the color-removal-related setting informationincludes the region information indicating a non-removal region or aremoval region. For example, the setting unit 1102 displays the previewimage on the screen of the operation unit 105 so that the graphic regiondetermined by the region determination unit 1103 is distinguishable bythe user on the screen. Then, the setting unit 1102 acquires the regioninformation modified based on a user input detected on the screen.

FIG. 12 illustrates an example of a GUI that can be displayed on thescreen of the operation unit 105 in the present embodiment. In the thirdembodiment, it is also assumed that the operation unit 105 displays, onthe screen, the color removal setting menu 940 having the check box 945exemplified in FIG. 9 and prompts the user to select whether or not todisplay the preview at the start of a job. When the execution of a copyjob is started with the preview display enabled, the operation unit 105displays, on a screen 1250, a preview image 1251 based on the colorimage data of the document read by the scanner 111 as illustrated on theleft of FIG. 12. In this example, in the preview image 1251, regions1252, 1253, 1254, and 1255 determined to be the graphic region aremasked so that they are distinguishable by the user. Note that, insteadof being concealed by masking, each region may be surrounded, forexample, by a line indicating the boundary of the region, or atranslucent object may be superimposed. These regions, if not cancelledby the user, are treated as non-removal regions in subsequent colorremoval processing. The operation unit 105 accepts cancellation oradditional designation of a non-removal region by the user on thisscreen. For example, cancellation of a non-removal region may beperformed by a touch operation to the region determined to be a graphicregion. An additional designation of a region may be performed using theabove two-point designation method or multi-point designation method. Onthe right of FIG. 12, a preview image 1251 modified by the user isillustrated, and a region 1252 is no longer a non-removal region.Meanwhile, a region 1256 has been newly and additionally designated as anon-removal region. The setting unit 1102 sets, as thecolor-removal-related setting information in the removal unit 1104, theregion information acquired as the result of modification of thenon-removal regions in the preview image.

(4) Removal Unit

Based on the region information set or specified by the setting unit1102, the removal unit 1104 removes the designated color from the colorimage data inputted from the data acquisition unit 1101 by replacing thecolor of the pixel indicating the designated color with the backgroundcolor (or converting the color of the pixel to a specific color). Asdescribed above, the background color may be, for example, white, acolor designated by the user, or an automatically-determined color.

For example, among the regions determined to be a graphic region by theregion determination unit 1103, the removal unit 1104 treats, as thenon-removal region, a region for which setting as the non-removal regionhas not been cancelled by the user and does not remove the designatedcolor in that region. Meanwhile, among the regions determined to be agraphic region by the region determination unit 1103, the removal unit1104 treats, as the removal region, a region for which setting as thenon-removal region has been cancelled by the user and removes thedesignated color in that region. In addition, among the regionsdetermined to be a graphic region, the removal unit 1104 treats, as thenon-removal region, a region for which designation as the non-removalregion has been added by the user and does not remove the designatedcolor in that region. Meanwhile, the removal unit 1104 treats, as aremoval region, a region that is not determined to be a graphic regionand for which designation as the non-removal region has not been addedand removes the designated color in that region. Such combination ofregion determination and user designation makes it possible toappropriately remove the color that should be removed from the imagedata of the document while reducing the burden of operation for regiondesignation on the user and reliably avoiding the execution of colorremoval that goes against the intention of the user.

<4-2. Processing Flow>

The flow of the basic processing performed by the multi-functionperipheral 100 in the third embodiment may be roughly the same as theflow described with reference to FIG. 6. However, in the thirdembodiment, after the document is read in step S613, at the beginning ofimage processing in S614, the display of the preview image and theacceptance of region information modification are performed by theoperation unit 105.

The flow of the image processing that corresponds to S614 of FIG. 6 inthe third embodiment may be roughly the same as the flow described withreference to FIG. 7A. However, in the third embodiment, as illustratedin FIG. 13, the region setting process is performed by the setting unit1102 after the region determination process in step S705. Further, inthe third embodiment, in step S733, it is determined whether or not thepixel of interest is a pixel in a non-removal region based on the resultof region determination by the region determination unit 1103 and theresult of region information modification by the user.

FIG. 13 is a flowchart illustrating an example of a detailed flow of aregion setting process that may be performed by the multi-functionperipheral 100 in the third embodiment. Here, the region setting processcan be realized, for example, by the CPU 101 executing a computerprogram loaded from the ROM 102 to the RAM 103 in cooperation with theimage processing unit 115.

In FIG. 13, it is illustrated that the region determination process(S705) described with reference to FIG. 7B by the region determinationunit 1103 prior to the region setting process is performed. Next, in51301, the setting unit 1102 displays a preview image on the screen ofthe operation unit 105 so that a region (i.e., a temporary non-removalregion) determined to be a graphic region as a result of the regiondetermination process is distinguishable on the screen. Next, in 51303,the operation unit 105 receives cancellation of a non-removal region oran additional designation of a non-removal region based on a user input,as described with reference to FIG. 12. Subsequent processing branchesin S1305 depending on whether the accepted operation represents acancellation of a non-removal region or an additional designation of anon-removal region. When the cancellation of the non-removal region hasbeen accepted, in S1307, the operation unit 105 cancels, on the screen,the masking of the designated non-removal region. In this case, thesetting for the region is modified from the non-removal region to theremoval region. Meanwhile, when the additional designation of thenon-removal region has been accepted, in 51309, the operation unit 105masks the designated region on the screen as a new non-removal region.In this case, the setting for the region is modified from the removalregion to the non-removal region. Such user interaction continues untilthe modification of the region information is completed (for example,the user touches the “OK” button) (S1311). When the modification of theregion information is completed, in S1313, the setting unit 1102acquires the modified region information as the color-removal-relatedsetting information and sets the acquired setting information in theremoval unit 1104. Then, the region setting process of FIG. 13 ends.

5. Summary

So far, several embodiments have been described in detail with referenceto FIGS. 1 to 13. In some embodiments, in the image processing apparatusin which a designated color is removed from the color image data of adocument by replacing the color of a pixel with a background color(converting the color of a pixel to a specific color), a graphic regionin the color image data is determined (specified), and the determinedgraphic region is excluded from a target of color removal (the colorconversion for the specified graphic region is skipped). According tothis configuration, with respect to the image data of the documentincluding the graphic region, it is possible to appropriately removefrom the image data the contents written in the designated color whilepreventing undesired deterioration in image quality.

In other embodiments, in the image processing apparatus in which adesignated color is removed from the color image data of a document byreplacing the color of a pixel with a background color (converting thecolor of a pixel to a specific color), a designation of a region by theuser on a preview image is accepted (the region is specified), and colorremoval (color conversion) is controlled based on that regiondesignation. By virtue of such a configuration, it is possible toappropriately remove a color that should be removed from image datawhile reliably avoiding the execution of color removal that goes againstthe explicit intention of the user.

In yet another embodiment, it is switched whether or not to performcolor removal (whether to skip color conversion) based on both thedetermination of the graphic region and the region designation by theuser. By virtue of such a configuration, if the user desires to remove aspecific color regardless of it being in a graphic region, for example,color removal can be performed as desired by the user. It is alsopossible to avoid performing color removal for regions that are notgraphic regions but which the user wishes to exclude from performingcolor removal.

6. Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of priority from Japanese PatentApplication No. 2021-021064, filed on Feb. 12, 2021 which is herebyincorporated by reference herein in its entirety.

What is claimed is:
 1. An image processing apparatus comprising: adesignating unit configured to designate a color; a reading unitconfigured to read a document to generate a document image; a conversionunit configured to convert a color of a pixel in the document image to aspecific color based on a designated color designated by the designatingunit; and a specification unit configured to specify a graphic region inthe document image; wherein the conversion unit is configured not toconvert a color of a pixel in the graphic region specified by thespecification unit to the specific color.
 2. The image processingapparatus according to claim 1, wherein the conversion unit isconfigured to convert, within the document image, a color of a pixel ofa character corresponding to the designated color to the specific colorwithout converting a color of a pixel in the graphic region to thespecific color.
 3. The image processing apparatus according to claim 1,wherein the conversion unit is configured to convert, within thedocument image, a color of a pixel outside the graphic region specifiedby the specification unit corresponding to the designated color to thespecific color without converting a color of a pixel in the graphicregion to the specific color.
 4. The image processing apparatusaccording to claim 1, wherein the specification unit is configured todetermine that a pixel of interest belongs to the graphic region in acase where a variance in color within a M-by-N pixel group (M and N areintegers larger than one) including the pixel of interest is lower thana predetermined criterion.
 5. The image processing apparatus accordingto claim 4, wherein the variance in color within the pixel group beinglower than the predetermined criterion includes, for any color componentvalue of any pixel of the pixel group, a difference between that colorcomponent value and a corresponding average color component valuecalculated over the pixel group being lower than a threshold.
 6. Theimage processing apparatus according to claim 4, wherein thespecification unit is configured to determine that a first pixel as apixel of interest and a second pixel as a neighboring pixel belong tothe graphic region in a case where the variance in color within a firstpixel group including the first pixel and the second pixel is lower thanthe predetermined criterion, and maintain the determination that thesecond pixel belongs to the graphic region regardless of a variance incolor within a second pixel group that is different from the first pixelgroup, the second pixel group including the second pixel.
 7. The imageprocessing apparatus according to claim 4, wherein M and N are set basedon reading resolution of the document.
 8. The image processing apparatusaccording to claim 1, wherein the specific color is white.
 9. The imageprocessing apparatus according to claim 1, further comprising: a userinterface configured to cause a preview image based on the documentimage to be displayed on a screen such that the graphic region isdistinguishable as a non-removal region by the user, wherein the userinterface is configured to accept a modification of the non-removalregion to a removal region by the user, and the conversion unit performsthe conversion to the specific color in the graphic region modified tothe removal region.
 10. The image processing apparatus according toclaim 9, wherein the user interface is configured to accept anadditional designation of a non-removal region by the user, and theconversion unit is configured not to perform the conversion to thespecific color in the non-removal region that has been designated. 11.The image processing apparatus according to claim 1, further comprising:a printing unit configured to print, on a sheet, an image for which theconversion to the specific color has been performed the conversion unit.12. An image processing method comprising: reading a document, by areader, to generate a document image; converting a color of a pixel inthe document image to a specific color based on a designated color;specifying a graphic region in the document image; and skipping theconversion a color of a pixel in the graphic region specified in thedocument image to the specific color.
 13. A computer-readable storagemedium having stored therein a computer program which, when executed bya processor of an image processing apparatus, causes the imageprocessing apparatus to perform operations comprising: reading adocument, by a reader, to generate a document image; converting a colorof a pixel in the document image to a specific color based on adesignated color; specifying a graphic region in the document image; andskipping the conversion a color of a pixel in the graphic regionspecified in the document image to the specific color.